python3 torchchat.py chat --help

usage: torchchat chat [-h] [--checkpoint-path CHECKPOINT_PATH] [--prompt PROMPT] [--chat] [--gui] [--num-samples NUM_SAMPLES] [--max-new-tokens MAX_NEW_TOKENS] [--top-k TOP_K] [--temperature TEMPERATURE] [--sequential-prefill] [--dso-path DSO_PATH] [--pte-path PTE_PATH] [--output-pte-path OUTPUT_PTE_PATH] [--output-dso-path OUTPUT_DSO_PATH] [--is-chat-model]
                      [--compile] [--compile-prefill] [--dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}] [--quantize QUANTIZE] [--device {fast,cpu,cuda,mps}] [--hf-token HF_TOKEN] [--model-directory MODEL_DIRECTORY] [--profile PROFILE] [--port PORT] [-v] [--seed SEED]
                      [model]

positional arguments:
  model                 Model name for well-known models

options:
  -h, --help            show this help message and exit
  --checkpoint-path CHECKPOINT_PATH
                        Use the specified model checkpoint path
  --is-chat-model       Indicate that the model was trained to support chat functionality
  --compile             Whether to compile the model with torch.compile
  --compile-prefill     Whether to compile the prefill. Improves prefill perf, but has higher compile times.
  --dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}
                        Override the dtype of the model (default is the checkpoint dtype). Options: bf16, fp16, fp32, fast16, fast
  --quantize QUANTIZE   Quantization options. pass in as '{"<mode>" : {"<argname1>" : <argval1>, "<argname2>" : <argval2>,...},}' modes are: embedding, linear:int8, linear:int4, linear:a8w4dq, precision.
  --device {fast,cpu,cuda,mps}
                        Hardware device to use. Options: cpu, cuda, mps
  --hf-token HF_TOKEN   A HuggingFace API token to use when downloading model artifacts
  --model-directory MODEL_DIRECTORY
                        The directory to store downloaded model artifacts. Default: /Users/jackkhuu/.torchchat/model-cache
  --profile PROFILE     Profile path.
  --port PORT           Port for the web server in browser mode
  -v, --verbose         Verbose output
  --seed SEED           Initialize torch seed

Generation Args:
  Configs for generating output based on provided prompt

  --prompt PROMPT       Input prompt for manual output generation
  --chat                Whether to start an interactive chat session
  --gui                 Whether to use a web UI for an interactive chat session
  --num-samples NUM_SAMPLES
                        Number of samples
  --max-new-tokens MAX_NEW_TOKENS
                        Maximum number of new tokens
  --top-k TOP_K         Top-k for sampling
  --temperature TEMPERATURE
                        Temperature for sampling
  --sequential-prefill  Whether to perform prefill sequentially. Only used for model debug.

Exported Model Path Args:
  Specify the path of the exported model files to ingest

  --dso-path DSO_PATH   Use the specified AOT Inductor .dso model file
  --pte-path PTE_PATH   Use the specified ExecuTorch .pte model file

Export Output Path Args:
  Specify the output path for the exported model files

  --output-pte-path OUTPUT_PTE_PATH
                        Output to the specified ExecuTorch .pte model file
  --output-dso-path OUTPUT_DSO_PATH
                        Output to the specified AOT Inductor .dso model file